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tell which qualities a reviewer used, and in what measure, in assessing a proposal. Although some agencies use such detailed ratings, others use a simple overall score.
A major new problem for peer review is the reduction in federal funding relative to the number of qualified applicants. This change is severely straining the system because of the difficulty of choosing wisely among very good proposals.
While the main basis for scientific accountability in the government-university relationship is preaward peer review, various kinds of postperformance review are also used. The simplest is the review of performance that occurs when requests for continued support are considered. Some agencies take this factor into account more explicitly than others. Postperformance review of a project can permit a judgment of whether best effort has been made, but judgment of the value of a basic research project must be made over a much longer time. Comparison of project outcomes to the original objectives may be interesting, but both positive and negative results are useful, and changes in objectives may mark the evolution of knowledge.
Postperformance review is used also to develop understanding of the factors affecting research productivity as a way of validating the basis of future funding choices.  Because the contributions of specific pieces of research are often not clear for a number of years, post-performance assessment of programs rather than projects, and over a longer time, can add another dimension to scientific accountability.  Similarly, postperformance review can profitably be done for an entire field. Again, the use of peer judgments is central.  It is important to recognize, however, that postperformance review consumes money, time, and effort, placing additional burdens on the agency and on the scientific enterprise.  A recent study by the Committee on Science, Engineering, and Public Policy30 investigated evaluation methods used by industry as well as by federal agencies. The study concluded that neither industry nor NSF has found anything better than some form of peer review for evaluating individual research projects.
Peer review, especially preaward peer review, serves more than scientific accountability.  It is an important vehicle of scientific communication and helps shape the direction of science.  The relative emphasis on criteria used in preperformance and postperformance reviews must be handled with care because they can easily become self-out the requirements of the revised A-21.
